Automated Essay Scoring 1 Automated Essay Scoring: Writing Assessment and Instruction

Automated Essay Scoring 2 Introduction This chapter documents the advent and rise of automated essay scoring (AES) as a means of both assessment and instruction. The first section discusses what AES is, how it works, and who the major purveyors of the technology are. The second section describes outgrowths of the technology as it applies to ongoing projects in measurement and education. In 1973, the late Ellis Page and colleagues at the University of Connecticut programmed the first successful automated essay scoring engine, " Project Essay Grade (PEG) " (1973). The technology was foretold some six years earlier in a landmark Phi Delta Kappan article entitled, " The Imminence of Grading Essays by Computer " (Page, 1966). At the time the article was provocative and a bit outrageous, though in hindsight, it can only be deemed prophetic. As a former high school English teacher , Page was convinced that students would benefit greatly by having access to technology that would provide quick feedback on their writing. He also realized that the greatest hindrance to having secondary students write more was the requirement that, ultimately, a teacher had to review stacks of papers. While PEG produced impressive results, the technology of the time was too primitive to make it a practical application. Text had to be typed on IBM 80-column punched cards and read into a mainframe computer before it could be evaluated. As a consequence, the technology sat dormant until the early 1990s and was revitalized with the confluence of two technological developments: microcomputers and the Internet. Microcomputers permitted the generation of electronic text from a regular keyboard and the internet provided a universal platform to submit text for review Automated Essay Scoring 3 Automated essay scoring is a measurement technology in which computers evaluate written work (Shermis & Burstein, 2003). Most of the initial applications have 2002). Computers do not " understand " the written text being evaluated. So, for example, the computer would not " get " the following joke. Q-When is a door not a door? A-When it is ajar. Unlike humans, a computer cannot interpret the play on words, and infer that the predicate in the answer (i.e., " ajar ") is being cleverly used as a noun (i.e., " a jar "). What the computer does in an AES context is to analyze the written text into its observable components. Different AES systems evaluate different …